*** Replication Materials for "Calibrating Confidence: Civic Education
*** and the Relationship Between Objective Political Knowledge
*** and Political Knowledge Confidence" ***

*** Last Updated July 11, 2024 ***


/// Getting Started /// 

use "jrs_perspectives_public_data_anon.dta"

tab obsType
*note there are 3 types of observations...
*1 = student completed wave 1 only
*2 = student completed wave 2 only
*3 = student completed both waves

*we will isolate our analysis to obsType==3 only
keep if obsType==3


/// Alphas and variable construction /// 

*knowledge confidence alphas
*reported in footnote 13
alpha k_parties_w1 k_elections_w1 k_constitution_w1 k_laws_w1 k_branches_w1
alpha k_parties_w2 k_elections_w2 k_constitution_w2 k_laws_w2 k_branches_w2

*create knowledge confidence scales
gen k_scale_w1=k_parties_w1+k_elections_w1+k_constitution_w1+k_laws_w1+k_branches_w1
gen k_scale_w2=k_parties_w2+k_elections_w2+k_constitution_w2+k_laws_w2+k_branches_w2

*put knowledge confience on a 0-5 scale for comparability
gen k_scale_w1a = (k_scale_w1/20)*5
gen k_scale_w2a = (k_scale_w2/20)*5

*label the variables
label variable k_scale_w1a "Knowldege Confidence w1 0/5"
label variable k_scale_w2a "Knowldege Confidence w2 0/5"

*create delta variable for knowledge confidence
gen delta_ka = k_scale_w2a - k_scale_w1a
label var delta_ka "Change in Knowl. Conf."

*objective knowledge alphas
*start with tetrachoric correlations
tetrachoric ok_parties_w1_correct ok_laws_w1_correct ok_branches_w1_correct ok_constitution_w1_correct ok_elections_w1_correct
tetrachoric ok_parties_w2_correct ok_laws_w2_correct ok_branches_w2_correct ok_constitution_w2_correct ok_elections_w2_correct 

* We then entered correlation table into Excel spreadsheet and calculated
* alpha using formula
* (k x average correlation) divided by (1 + average correlation * k-1)
* where k = number of items (5)
* this spreadsheet is provided with data files

*factor analysis to verify fit
*factor scores reported in Part E of Online Supplemental Material
factor ok_parties_w1_correct ok_laws_w1_correct ok_branches_w1_correct ok_constitution_w1_correct ok_elections_w1_correct 
factor ok_parties_w2_correct ok_laws_w2_correct ok_branches_w2_correct ok_constitution_w2_correct ok_elections_w2_correct

*create objective knowledge scales
gen ok_correct_w1 = ok_parties_w1_correct + ok_laws_w1_correct + ok_branches_w1_correct + ok_constitution_w1_correct + ok_elections_w1_correct
gen ok_correct_w2 = ok_parties_w2_correct + ok_laws_w2_correct + ok_branches_w2_correct + ok_constitution_w2_correct + ok_elections_w2_correct
label variable ok_correct_w1 "Objective Knowledge: Total Correct at Beginning of Semester"
label variable ok_correct_w2 "Objective Knowledge: Total Correct at End of Semester"

*generate a collapsed version of wave 1 objective knowledge due to few responses on low end of scale
* 0 = 0, 1, or 2 objective Qs correct
* 1 = 3 Qs correct
* 2 = 4 Qs correct
* 3 = 5 Qs correct
gen ok_w1_3 = 0 if ok_correct_w1 <= 2
recode ok_w1_3 . = 1 if ok_correct_w1 == 3
recode ok_w1_3 . = 2 if ok_correct_w1 == 4
recode ok_w1_3 . = 3 if ok_correct_w1 == 5
label define ok_w1 0 "≤2/5 Qs Correct" 1 "3/5 Qs Correct" 2 "4/5 Qs Correct" 3 "5/5 Qs Correct" 
label values ok_w1_3 ok_w1

*create delta var for objective knowledge
gen delta_ok = ok_correct_w2 - ok_correct_w1
label var delta_ok "Change in Obj. Knowl."

* gen directional gap between subjective and objective knowledge 
	* (subjective - objective), using 0-5 dummy scale for both
gen subjObj_gap_w1_5 = k_scale_w1a - ok_correct_w1
gen subjObj_gap_w2_5 = k_scale_w2a - ok_correct_w2
label variable subjObj_gap_w1_5 "Directional Confidence Gap w1"
label variable subjObj_gap_w2_5 "Directional Confidence Gap w2"

* gen absolute gap between subjective and objective knowledge
gen abs_subjObj_gap_w1_5 = abs(subjObj_gap_w1_5)
gen abs_subjObj_gap_w2_5 = abs(subjObj_gap_w2_5)
label variable abs_subjObj_gap_w1_5 "Absolute Confidence Gap w1"
label variable abs_subjObj_gap_w2_5 "Absolute Confidence Gap w2"

*create indicator for overconfidence
gen overconfident = 1 if subjObj_gap_w1_5 >0
recode overconfident . = 0 if subjObj_gap_w1_5<=0
label define overconflabel 0 "Underconf." 1 "Overconf."
label values overconfident overconflabel


/// Descriptive Statistics for Dataset /// 
*provides N respondents by section
tab crn_final
*provides demographics of respondents
tab female 
tab minority 
tab year_w1 
tab major_w1 
*provides distribution of objective knowledge at start of semester
tab ok_w1_3 


/// Analysis ///

*compare objective knowledge at w1 and w2
*reported in Figure 1
ttest ok_correct_w1 == ok_correct_w2

*compare objective knowledge at w1 and w2 by starting level of obj. knowl.
*reported in Figure 3
by ok_w1_3, sort : ttest ok_correct_w1 == ok_correct_w2

*compare knowledge confidence w1 and w2
*reported in Figure 1
ttest k_scale_w1a == k_scale_w2a

*compare knowledge conf. at w1 and w2 by starting level of obj. knowl.
*reported in Figure 3
by ok_w1_3, sort : ttest k_scale_w1a == k_scale_w2a

*create Figure 1 Showing Objective Knowledge and Confidence at both waves
*first create bar graph of objective knowledge
graph bar (mean) ok_correct_w1 (mean) ok_correct_w2 if obsType==3, legend(order(1 "Wave 1" 2  "Wave 2")) scheme(sj) graphregion(fcolor(white) lcolor(white)) title(Objective Knowledge)

*then save but replace location below with a path to your working directory and/or unique file name to be created in your working directory
graph save "Graph" "Objective.gph", replace

*next create bar graph of knowledge confidence
graph bar (mean) k_scale_w1a (mean) k_scale_w2a if obsType==3, ylabel(, gmax) legend(order(1 "Wave 1" 2  "Wave 2")) scheme(sj) graphregion(fcolor(white) lcolor(white)) title(Knowledge Confidence) 

*then save but replace location below with a path to your working directory and/or unique file name to be created in your working directory
graph save "Graph" "Confidence.gph", replace

*finally combine into single graph replicating Figure 1
graph combine "Objective.gph" "Confidence.gph", graphregion(fcolor(white) lcolor(white)) plotregion(lcolor(white))

*changes in directional subjective-objective gap
*first, aggregate comparison testing whether directional gap closes between waves
ttest subjObj_gap_w1_5 == subjObj_gap_w2_5

*create Figure 2 showing aggregate comparison of subj-obj gap between the two waves
*first create absolute gap graph
graph bar (mean) abs_subjObj_gap_w1_5 (mean) abs_subjObj_gap_w2_5 if obsType==3, legend(order(1 "Wave 1" 2  "Wave 2")) scheme(sj) graphregion(fcolor(white) lcolor(white)) title(Absolute Confidence Gap)

*then save but replace location below with a path to your working directory and/or unique file name to be created in your working directory
graph save "Graph" "AbsoluteGap.gph", replace

*next create direction gap graph
graph bar (mean) subjObj_gap_w1_5 (mean) subjObj_gap_w2_5 if obsType==3, legend(order(1 "Wave 1" 2  "Wave 2")) scheme(sj) graphregion(fcolor(white) lcolor(white)) title(Directional Confidence Gap)

*then save but replace location below with a path to your working directory and/or unique file name to be created in your working directory
graph save "Graph" "DirectionalGap.gph", replace

*finally combine to replicate Figure 2
graph combine "AbsoluteGap.gph" "DirectionalGap.gph", graphregion(fcolor(white) lcolor(white)) plotregion(lcolor(white))

*create Figure 3 Comparing objective know and know conf for each wave by w1 objective knowledge
graph bar (mean) ok_correct_w1 (mean) ok_correct_w2 (mean) k_scale_w1a (mean) k_scale_w2a if obsType==3, legend(order(1 "Wave 1 Obj. Knowledge" 2  "Wave 2 Obj. Knowledge" 3 "Wave 1 Knowledge Conf." 4 "Wave 2 Knowledge Conf.") rows(2)) scheme(sj) by(, note("") graphregion(fcolor(white) lcolor(white))) by(ok_w1_3) 
	
*compare avg change in obj and change in subj across collapsed levels of w1 obj
by ok_w1_3, sort : ttest delta_ka == delta_ok

*compare avg change in obj for those in low vs. high collapsed levels of w1 obj
ttest delta_ok if obsType==3 & ok_w1_3==0 | ok_w1_3==3, by(ok_w1_3)

*compare avg change in subj for those in low vs. high collapsed levels of w1 obj
ttest delta_ka if obsType==3 & ok_w1_3==0 | ok_w1_3==3, by(ok_w1_3)

*create Figure 4 showing absolute gap by obj. knowl. at wave 1
graph bar (mean) abs_subjObj_gap_w1_5 (mean) abs_subjObj_gap_w2_5 if obsType==3, legend(order(1 "Wave 1" 2  "Wave 2")) scheme(sj) by(, note("") graphregion(fcolor(white) lcolor(white))) by(ok_w1_3)

by ok_w1_3, sort : ttest abs_subjObj_gap_w1_5 == abs_subjObj_gap_w2_5 

*also can compare change in subj-obj gap by collapsed levels of w1 obj	
*create change in gap variable
gen delta_abs_subjObj_gap_5 = abs_subjObj_gap_w1_5-abs_subjObj_gap_w2_5
label variable delta_abs_subjObj_gap_5 "Change in Abs. Confidence Gap w1-w2"
*then t-tests
ttest delta_abs_subjObj_gap_5 if obsType==3 & ok_w1_3==0 | ok_w1_3==3, by(ok_w1_3)
ttest delta_abs_subjObj_gap_5 if obsType==3 & ok_w1_3==1 | ok_w1_3==3, by(ok_w1_3)
ttest delta_abs_subjObj_gap_5 if obsType==3 & ok_w1_3==2 | ok_w1_3==3, by(ok_w1_3)

*create Figure 5 showing directional subj-obj gap by obj. knowl. at wave 1
graph bar (mean) subjObj_gap_w1_5 (mean) subjObj_gap_w2_5 if obsType==3, legend(order(1 "Wave 1" 2  "Wave 2")) scheme(sj) by(, note("") graphregion(fcolor(white) lcolor(white))) by(ok_w1_3)

*compare subj-obj gap across collapsed levels of w1 obj
by ok_w1_3, sort : ttest subjObj_gap_w1_5 == subjObj_gap_w2_5 

* A closer look at those with the lowest level of objective knowledge
* We track how many move from being overconfident to being underconfident or calibrated
gen overconfident_w2 = 0 
replace overconfident_w2 = 1 if subjObj_gap_w2_5 <= 0 
label values overconfident_w2 overconflabel
tab overconfident overconfident_w2 if ok_w1_3==0

*For footnote 14 re: relatively high political knowledge in wave 1
sum ok_elections_w1_correct ok_laws_w1_correct ok_branches_w1_correct ///
	ok_parties_w1_correct ok_constitution_w1_correct

	
// Appendix Figures and Models //

**Part B: Distribution of Objective Knowledge at Wave 1 and Wave 2**

*Figure A1
histogram ok_correct_w1 if obsType==3, discrete fraction scheme(sj) graphregion(fcolor(white) lcolor(white)) ytitle("Proportion of Observations")
histogram ok_correct_w2 if obsType==3, discrete fraction scheme(sj) graphregion(fcolor(white) lcolor(white)) ytitle("Proportion of Observations")

**Part C: Relationship between Objective Knowledge and Knowledge Confidence by Item**

*Figure A2
*note additional editing done in Graph Editor to fix labels and set colors
graph bar (mean) k_parties_w1 (mean) k_laws_w1 (mean) k_branches_w1 (mean) k_constitution_w1 (mean) k_elections_w1, scheme(sj)
graph bar (mean) k_parties_w2 (mean) k_laws_w2 (mean) k_branches_w2 (mean) k_constitution_w2 (mean) k_elections_w2, scheme(sj)
	
*Figure A3
graph bar (mean) ok_parties_w2_correct  (mean) ok_laws_w2_correct  (mean) ok_branches_w2_correct  (mean) ok_constitution_w2_correct  (mean) ok_elections_w2_correct, scheme(sj)  
graph bar (mean) ok_parties_w1_correct  (mean) ok_laws_w1_correct (mean) ok_branches_w1_correct  (mean) ok_constitution_w1_correct   (mean) ok_elections_w1_correct, scheme(sj)

*Table A1a-j 
*distribution of knowledge confidence by objective knowledge for each item
*row percentages and chi-squared statistic included
*non-random distribution of knowledge confidence (higher for those who got it correct) 
*for all items except elections
tab ok_parties_w1_correct k_parties_w1 if obsType==3, row chi2
tab ok_parties_w2_correct k_parties_w2 if obsType==3, row chi2
tab ok_elections_w1_correct k_elections_w1 if obsType==3, row chi2
tab ok_elections_w2_correct k_elections_w2 if obsType==3, row chi2
tab ok_branches_w2_correct k_branches_w2 if obsType==3, row chi2
tab ok_branches_w1_correct k_branches_w1 if obsType==3, row chi2
tab ok_laws_w1_correct k_laws_w1 if obsType==3, row chi2
tab ok_laws_w2_correct k_laws_w2 if obsType==3, row chi2
tab ok_constitution_w2_correct k_constitution_w2 if obsType==3, row chi2
tab ok_constitution_w1_correct k_constitution_w1 if obsType==3, row chi2

**Part D: Means and Statistical Significance of Differences from Wave 1 to Wave 2**
*Table A2	
ttest ok_correct_w1 == ok_correct_w2
ttest k_scale_w1a == k_scale_w2a
ttest subjObj_gap_w1_5 == subjObj_gap_w2_5
ttest abs_subjObj_gap_w1_5 == abs_subjObj_gap_w2_5
		
**Part E: Factor Analysis of Knowledge Items**
*code provided on lines 59-60 above

**Part F: Linear Regression Models of the Confidence Gap**
*prepare data and packages
encode semester, gen(sem)
ssc install estout

*descriptive statistics
*Table A5
tab ok_w1_3 overconfident

*Table A6

	* Model 1

reg subjObj_gap_w2_5 delta_ka delta_ok ok_w1_3 overconfident courseAttendance_w2 parentEdu_w1 year_w1 gpa_combo female minority i.instructor_final i.sem 
eststo m1

	* Model 2

reg subjObj_gap_w2_5 c.delta_ka##c.ok_w1_3 c.delta_ok##c.ok_w1_3 overconfident courseAttendance_w2 parentEdu_w1 year_w1 gpa_combo female minority i.instructor_final i.sem 
eststo m2

	* Model 3 with 3way interaction

reg subjObj_gap_w2_5 c.delta_ka##i.overconfident##c.ok_w1_3 c.delta_ok##i.overconfident##c.ok_w1_3 courseAttendance_w2 parentEdu_w1 year_w1 gpa_combo female minority i.instructor_final i.sem
eststo m3

	*create table
	*note: additional editing done in Microsoft Word
esttab m1 m2 m3, b(2) se(2) ar2 bic star(+ 0.1 * 0.05)

*Figure A4
*obtain ranges of delta_ok and delta_ka by objective knowl wave 1
ssc install fre
by ok_w1_3, sort : fre delta_ok delta_ka
*use ranges of variables to make plots below

*calculate and plot marginal effect for  delta_ka
margins overconfident, at(delta_ka=(-1.25(0.25)4.5) ok_w1_3=0) atmeans saving(dka0, replace)

marginsplot, recast(connected) recastci(rcap) level(95) title("≤2/5 Qs Correct") yscale(r(-3 6)) ytitle("") xscale(r(-1.25 4.5)) xlabel(minmax) legend(cols(1))  yline(0) plotregion(fcolor(white) lcolor(gs10) lwidth(med)) addplot(hist delta_ka if ok_w1_3==0, recast(area) blcolor(black) fcolor(black%25) percent yaxis(2) yscale(alt lcolor() axis(2)) ylabel(0 "0%" 25 "25%" 50 "50%" 75 "75%" 100 "100%", labcolor() axis(2) tlcolor(black) tlwidth(thin) labsize(small) tl(0)) ytitle(" ", axis(2))) scheme(s1mono)

graph save "Graph" "dka0.gph", replace

	* note: y-axis needs to be manually adjusted to go from -3, 6 in graph editor
margins overconfident, at(delta_ka=(-2.5(0.25)3.25) ok_w1_3=1) atmeans saving(dka1, replace)

marginsplot, recast(connected) recastci(rcap) level(95) title("3/5 Qs Correct") yscale(r(-3 6)) ytitle("") xscale(r(-2.5 3.25)) xlabel(minmax) legend(cols(1)) yline(0) plotregion(fcolor(white) lcolor(gs10) lwidth(med)) addplot(hist delta_ka if ok_w1_3==1, recast(area) blcolor(black) fcolor(black%25) percent yaxis(2) yscale(alt lcolor() axis(2)) ylabel(0 "0%" 25 "25%" 50 "50%" 75 "75%" 100 "100%", labcolor() axis(2) tlcolor(black) tlwidth(thin) labsize(small) tl(0)) ytitle(" ", axis(2))) scheme(s1mono)

graph save "Graph" "dka1.gph", replace

margins overconfident, at(delta_ka=(-2(0.25)4) ok_w1_3=2) atmeans saving(dka2, replace)

marginsplot, recast(connected) recastci(rcap) level(95) title("4/5 Qs Correct") yscale(r(-3 6)) ytitle("") xscale(r(-2 4)) xlabel(minmax) legend(cols(1))  yline(0) plotregion(fcolor(white) lcolor(gs10) lwidth(med)) addplot(hist delta_ka if ok_w1_3==2, recast(area) blcolor(black) fcolor(black%25) percent yaxis(2) yscale(alt lcolor() axis(2)) ylabel(0 "0%" 25 "25%" 50 "50%" 75 "75%" 100 "100%", labcolor() axis(2) tlcolor(black) tlwidth(thin) labsize(small) tl(0)) ytitle(" ", axis(2))) scheme(s1mono)

graph save "Graph" "dka2.gph", replace

	*note: overconfident plot hidden in graph editor since substantively, 5/5 respondents cannot be overconfident
margins overconfident, at(delta_ka=(-1.75(0.25)3.5) ok_w1_3=3) atmeans saving(dka3, replace)

marginsplot, recast(connected) recastci(rcap) level(95) title("5/5 Qs Correct") yscale(r(-3 6)) ytitle("") xscale(r(-1.75 3.5)) xlabel(minmax) legend(cols(1))  yline(0) plotregion(fcolor(white) lcolor(gs10) lwidth(med)) addplot(hist delta_ka if ok_w1_3==3, recast(area) blcolor(black) fcolor(black%25) percent yaxis(2) yscale(alt lcolor() axis(2)) ylabel(0 "0%" 25 "25%" 50 "50%" 75 "75%" 100 "100%", labcolor() axis(2) tlcolor(black) tlwidth(thin) labsize(small) tl(0)) ytitle(" ", axis(2))) scheme(s1mono)

graph save "Graph" "dka3.gph", replace

*combine graphs 
	*note: some additional small edits for clarity done using graph editor and
	*y-axis may need to be manually adjusted to go from -3, 6 in graph editor
graph combine "dka0" "dka1" "dka2" "dka3", ycom cols(4) cap("y-axis 1 = predicted value of confidence gap; y-axis 2 = percent of observations across values of change in knowl. conf.", size(vsmall)) scheme(s1mono)

graph save "Graph" "dka final.gph", replace

*Figure A5
*calculate and plot marginal effect for delta_ok
margins overconfident, at(delta_ok=(-2(1)4) ok_w1_3=0) atmeans saving(dok1, replace)

marginsplot, recast(connected) recastci(rcap) level(95) title("≤2/5 Qs Correct") yscale(r(-4(2)6)) ytitle("") xscale(r(-2 4)) xlabel(minmax) legend(cols(1))  yline(0) plotregion(fcolor(white) lcolor(gs10) lwidth(med)) addplot(hist delta_ok if ok_w1_3==0, recast(area) blcolor(black) fcolor(black%25) percent yaxis(2) yscale(alt lcolor() axis(2)) ylabel(0 "0%" 25 "25%" 50 "50%" 75 "75%" 100 "100%", labcolor() axis(2) tlcolor(black) tlwidth(thin) labsize(small) tl(0)) ytitle(" ", axis(2))) scheme(s1mono)

graph save "Graph" "dok0.gph", replace

margins overconfident, at(delta_ok=(-3(1)2) ok_w1_3=1) atmeans saving(dok1, replace)

marginsplot, recast(connected) recastci(rcap) level(95) title("3/5 Qs Correct") yscale(r(-4(2)6)) ytitle("") xscale(r(-3 2)) xlabel(minmax) legend(cols(1))  yline(0) plotregion(fcolor(white) lcolor(gs10) lwidth(med)) addplot(hist delta_ok if ok_w1_3==1, recast(area) blcolor(black) fcolor(black%25) percent yaxis(2) yscale(alt lcolor() axis(2)) ylabel(0 "0%" 25 "25%" 50 "50%" 75 "75%" 100 "100%", labcolor() axis(2) tlcolor(black) tlwidth(thin) labsize(small) tl(0)) ytitle(" ", axis(2))) scheme(s1mono)

graph save "Graph" "dok1.gph", replace

margins overconfident, at(delta_ok=(-4(1)1) ok_w1_3=2) atmeans saving(dok1, replace)

marginsplot, recast(connected) recastci(rcap) level(95) title("4/5 Qs Correct") yscale(r(-4 6)) ytitle("") xscale(r(-4 1)) xlabel(minmax) legend(cols(1))  yline(0) plotregion(fcolor(white) lcolor(gs10) lwidth(med)) addplot(hist delta_ok if ok_w1_3==2, recast(area) blcolor(black) fcolor(black%25) percent yaxis(2) yscale(alt lcolor() axis(2)) ylabel(0 "0%" 25 "25%" 50 "50%" 75 "75%" 100 "100%", labcolor() axis(2) tlcolor(black) tlwidth(thin) labsize(small) tl(0)) ytitle(" ", axis(2))) scheme(s1mono)

graph save "Graph" "dok2.gph", replace

	* overconfident plot hidden in graph editor since substantively, 5/5 respondents cannot be overconfident
margins overconfident, at(delta_ok=(-4(1)0) ok_w1_3=3) atmeans saving(dok1, replace)

marginsplot, recast(connected) recastci(rcap) level(95) title("5/5 Qs Correct") yscale(r(-4 6)) ytitle("") xscale(r(-4 0)) xlabel(minmax) legend(cols(1))  yline(0) plotregion(fcolor(white) lcolor(gs10) lwidth(med)) addplot(hist delta_ok if ok_w1_3==3, recast(area) blcolor(black) fcolor(black%25) percent yaxis(2) yscale(alt lcolor() axis(2)) ylabel(0 "0%" 25 "25%" 50 "50%" 75 "75%" 100 "100%", labcolor() axis(2) tlcolor(black) tlwidth(thin) labsize(small) tl(0)) ytitle(" ", axis(2))) scheme(s1mono)

graph save "Graph" "dok3.gph", replace

*combine graphs
*note: some additional editing done for clarity using graph editor
* note: y-axis may need to be manually adjusted to go from -4, 6 in graph editor
graph combine "dok0" "dok1" "dok2" "dok3", ycom cols(4) cap("y-axis 1 = predicted value of confidence gap; y-axis 2 = percent of observations across values of change in knowl. conf.", size(vsmall)) scheme(s1mono)

graph save "Graph" "dok final.gph", replace


*Repeat for absolute confidence gap
*Table A7

	* Model 1

reg abs_subjObj_gap_w2_5 delta_ka delta_ok ok_w1_3 overconfident courseAttendance_w2 parentEdu_w1 year_w1 gpa_combo female minority i.instructor_final i.sem 

eststo m1abs

	* Model 2

reg abs_subjObj_gap_w2_5 c.delta_ka##c.ok_w1_3 c.delta_ok##c.ok_w1_3 overconfident courseAttendance_w2 parentEdu_w1 year_w1 gpa_combo female minority i.instructor_final i.sem 

eststo m2abs

	* Model 3 w/ 3 way interaction

reg abs_subjObj_gap_w2_5 c.delta_ka##i.overconfident##c.ok_w1_3 c.delta_ok##i.overconfident##c.ok_w1_3 courseAttendance_w2 parentEdu_w1 year_w1 gpa_combo female minority i.instructor_final i.sem

eststo m3abs

	*create table
	*note: additional editing done in Microsoft Word
esttab m1abs m2abs m3abs, b(2) se(2) ar2 bic star(+ 0.1 * 0.05)

*Figure A6 
margins overconfident, at(delta_ka=(-1.25(0.25)4.5) ok_w1_3=0) atmeans saving(dka0, replace)

marginsplot, recast(connected) recastci(rcap) level(95) title("≤2/5 Qs Correct") yscale(r(0 6)) ytitle("") xscale(r(-1.25 4.5)) xlabel(minmax) legend(cols(1))  yline(0) plotregion(fcolor(white) lcolor(gs10) lwidth(med)) addplot(hist delta_ka if ok_w1_3==0, recast(area) blcolor(black) fcolor(black%25) percent yaxis(2) yscale(alt lcolor() axis(2)) ylabel(0 "0%" 25 "25%" 50 "50%" 75 "75%" 100 "100%", labcolor() axis(2) tlcolor(black) tlwidth(thin) labsize(small) tl(0)) ytitle(" ", axis(2))) scheme(s1mono)

graph save "Graph" "dka0abs.gph", replace

	* note: y-axis needs to be manually adjusted to go from -3, 6 in graph editor
margins overconfident, at(delta_ka=(-2.5(0.25)3.25) ok_w1_3=1) atmeans saving(dka1, replace)

marginsplot, recast(connected) recastci(rcap) level(95) title("3/5 Qs Correct") yscale(r(0 6)) ytitle("") xscale(r(-2.5 3.25)) xlabel(minmax) legend(cols(1)) yline(0) plotregion(fcolor(white) lcolor(gs10) lwidth(med)) addplot(hist delta_ka if ok_w1_3==1, recast(area) blcolor(black) fcolor(black%25) percent yaxis(2) yscale(alt lcolor() axis(2)) ylabel(0 "0%" 25 "25%" 50 "50%" 75 "75%" 100 "100%", labcolor() axis(2) tlcolor(black) tlwidth(thin) labsize(small) tl(0)) ytitle(" ", axis(2))) scheme(s1mono)

graph save "Graph" "dka1abs.gph", replace

margins overconfident, at(delta_ka=(-2(0.25)4) ok_w1_3=2) atmeans saving(dka2, replace)

marginsplot, recast(connected) recastci(rcap) level(95) title("4/5 Qs Correct") yscale(r(0 6)) ytitle("") xscale(r(-2 4)) xlabel(minmax) legend(cols(1))  yline(0) plotregion(fcolor(white) lcolor(gs10) lwidth(med)) addplot(hist delta_ka if ok_w1_3==2, recast(area) blcolor(black) fcolor(black%25) percent yaxis(2) yscale(alt lcolor() axis(2)) ylabel(0 "0%" 25 "25%" 50 "50%" 75 "75%" 100 "100%", labcolor() axis(2) tlcolor(black) tlwidth(thin) labsize(small) tl(0)) ytitle(" ", axis(2))) scheme(s1mono)

graph save "Graph" "dka2abs.gph", replace

* overconfident plot hidden in graph editor since substantively, 5/5 respondents cannot be overconfident
margins overconfident, at(delta_ka=(-1.75(0.25)3.5) ok_w1_3=3) atmeans saving(dka3, replace)

marginsplot, recast(connected) recastci(rcap) level(95) title("5/5 Qs Correct") yscale(r(0 6)) ytitle("") xscale(r(-1.75 3.5)) xlabel(minmax) legend(cols(1))  yline(0) plotregion(fcolor(white) lcolor(gs10) lwidth(med)) addplot(hist delta_ka if ok_w1_3==3, recast(area) blcolor(black) fcolor(black%25) percent yaxis(2) yscale(alt lcolor() axis(2)) ylabel(0 "0%" 25 "25%" 50 "50%" 75 "75%" 100 "100%", labcolor() axis(2) tlcolor(black) tlwidth(thin) labsize(small) tl(0)) ytitle(" ", axis(2))) scheme(s1mono)

graph save "Graph" "dka3abs.gph", replace

*combine graphs
*note: some additional small edits for clarity done using graph editor
* note: y-axis may need to be manually adjusted to go from -3, 6 in graph editor
graph combine "dka0abs" "dka1abs" "dka2abs" "dka3abs", ycom cols(4) cap("y-axis 1 = predicted value of abs. conf. gap; y-axis 2 = percent of observations across values of change in knowl. conf.", size(vsmall)) scheme(s1mono)

graph save "Graph" "dka final abs.gph", replace

*Figure A7
*for obj. knowledge changes
margins overconfident, at(delta_ok=(-2(1)4) ok_w1_3=0) atmeans saving(dok1, replace)

marginsplot, recast(connected) recastci(rcap) level(95) title("≤2/5 Qs Correct") yscale(r(0(1)6)) ytitle("") xscale(r(-2 4)) xlabel(minmax) legend(cols(1))  yline(0) plotregion(fcolor(white) lcolor(gs10) lwidth(med)) addplot(hist delta_ok if ok_w1_3==0, recast(area) blcolor(black) fcolor(black%25) percent yaxis(2) yscale(alt lcolor() axis(2)) ylabel(0 "0%" 25 "25%" 50 "50%" 75 "75%" 100 "100%", labcolor() axis(2) tlcolor(black) tlwidth(thin) labsize(small) tl(0)) ytitle(" ", axis(2))) scheme(s1mono)

graph save "Graph" "dok0abs.gph", replace

margins overconfident, at(delta_ok=(-3(1)2) ok_w1_3=1) atmeans saving(dok1, replace)

marginsplot, recast(connected) recastci(rcap) level(95) title("3/5 Qs Correct") yscale(r(0(1)6)) ytitle("") xscale(r(-3 2)) xlabel(minmax) legend(cols(1))  yline(0) plotregion(fcolor(white) lcolor(gs10) lwidth(med)) addplot(hist delta_ok if ok_w1_3==1, recast(area) blcolor(black) fcolor(black%25) percent yaxis(2) yscale(alt lcolor() axis(2)) ylabel(0 "0%" 25 "25%" 50 "50%" 75 "75%" 100 "100%", labcolor() axis(2) tlcolor(black) tlwidth(thin) labsize(small) tl(0)) ytitle(" ", axis(2))) scheme(s1mono)

graph save "Graph" "dok1abs.gph", replace

margins overconfident, at(delta_ok=(-4(1)1) ok_w1_3=2) atmeans saving(dok1, replace)

marginsplot, recast(connected) recastci(rcap) level(95) title("4/5 Qs Correct") yscale(r(0(1)6)) ytitle("") xscale(r(-4 1)) xlabel(minmax) legend(cols(1))  yline(0) plotregion(fcolor(white) lcolor(gs10) lwidth(med)) addplot(hist delta_ok if ok_w1_3==2, recast(area) blcolor(black) fcolor(black%25) percent yaxis(2) yscale(alt lcolor() axis(2)) ylabel(0 "0%" 25 "25%" 50 "50%" 75 "75%" 100 "100%", labcolor() axis(2) tlcolor(black) tlwidth(thin) labsize(small) tl(0)) ytitle(" ", axis(2))) scheme(s1mono)

graph save "Graph" "dok2abs.gph", replace

	* overconfident plot hidden in graph editor since substantively, 5/5 respondents cannot be overconfident
margins overconfident, at(delta_ok=(-4(1)0) ok_w1_3=3) atmeans saving(dok1, replace)
marginsplot, recast(connected) recastci(rcap) level(95) title("5/5 Qs Correct") yscale(r(0(1)6)) ytitle("") xscale(r(-4 0)) xlabel(minmax) legend(cols(1))  yline(0) plotregion(fcolor(white) lcolor(gs10) lwidth(med)) addplot(hist delta_ok if ok_w1_3==3, recast(area) blcolor(black) fcolor(black%25) percent yaxis(2) yscale(alt lcolor() axis(2)) ylabel(0 "0%" 25 "25%" 50 "50%" 75 "75%" 100 "100%", labcolor() axis(2) tlcolor(black) tlwidth(thin) labsize(small) tl(0)) ytitle(" ", axis(2))) scheme(s1mono)

graph save "Graph" "dok3abs.gph", replace

*combine graphs together
*note: some additional editing done for clarity using graph editor
* note: y-axis may need to be manually adjusted to go from -4, 6 in graph editor
graph combine "dok0abs" "dok1abs" "dok2abs" "dok3abs", ycom cols(4) cap("y-axis 1 = predicted value of abs. conf. gap; y-axis 2 = percent of observations across values of change in knowl. conf.", size(vsmall)) scheme(s1mono)

graph save "Graph" "dok final abs.gph", replace

**Part G: Using Overconfidence as the Primary Moderator Variable
*some moderate correlations between versions of variables
corr subjObj_gap_w2_5 delta_ka delta_ok ok_w1_3 overconfident

*rerun base model and obtain vifs
reg subjObj_gap_w2_5 delta_ka delta_ok ok_w1_3 overconfident courseAttendance_w2 parentEdu_w1 year_w1 gpa_combo female minority i.instructor_final i.sem 
vif, uncentered
*no vifs over 10 for key variables
*though there is some significant multicollinearity between control variables

*rerun models using overconfident as primary moderator variable to reduce multicollinearity
reg subjObj_gap_w2_5 delta_ka delta_ok overconfident courseAttendance_w2 parentEdu_w1 year_w1 gpa_combo female minority i.instructor_final i.sem 
eststo m1_no_ok

reg subjObj_gap_w2_5 c.delta_ka##i.overconfident c.delta_ok##i.overconfident courseAttendance_w2 parentEdu_w1 year_w1 gpa_combo female minority i.instructor_final i.sem
eststo m3_no_ok

reg abs_subjObj_gap_w2_5 delta_ka delta_ok overconfident courseAttendance_w2 parentEdu_w1 year_w1 gpa_combo female minority i.instructor_final i.sem 
eststo m1_no_ok_abs

reg abs_subjObj_gap_w2_5 c.delta_ka##i.overconfident c.delta_ok##i.overconfident courseAttendance_w2 parentEdu_w1 year_w1 gpa_combo female minority i.instructor_final i.sem
eststo m3_no_ok_abs

*Table A9
esttab m1_no_ok m3_no_ok m1_no_ok_abs m3_no_ok_abs, b(2) se(2) ar2 bic star(+ 0.1 * 0.05)

*also see if ttests replicate using overconfidence instead of objective knowledge
*results of these tests reported in Table A8

*directional gap reduces for both groups
by overconfident, sort : ttest subjObj_gap_w1_5 == subjObj_gap_w2_5 if obsType==3

*absolute gaps
by overconfident, sort : ttest abs_subjObj_gap_w1_5 == abs_subjObj_gap_w2_5 if obsType==3

*but due to significant gains in obj. knowledge for the underconfident
*no significant gain in obj. knowledge for the overconfident
by overconfident, sort : ttest ok_correct_w1 == ok_correct_w2 if obsType==3

*and gains in knowledge confidence for both
by overconfident, sort : ttest k_scale_w1a == k_scale_w2a if obsType==3


**Part H: Analyzing the Confidence Gap by Topic**
*first place knowledge confidence for each topic on a 0 1 scale
gen k_parties_w1_01 = k_parties_w1/4
gen k_laws_w1_01 = k_laws_w1/4
gen k_constitution_w1_01 = k_constitution_w1/4
gen k_elections_w1_01 = k_elections_w1/4
gen k_branches_w1_01 = k_branches_w1/4
gen k_parties_w2_01 = k_parties_w2/4
gen k_laws_w2_01 = k_laws_w2/4
gen k_constitution_w2_01 = k_constitution_w2/4
gen k_elections_w2_01 = k_elections_w2/4
gen k_branches_w2_01 = k_branches_w2/4

*then generate gap measure using gaps for each item
gen subjObj_gap_w1_parties = (k_parties_w1_01 - ok_parties_w1_correct)
gen subjObj_gap_w2_parties = (k_parties_w2_01 - ok_parties_w2_correct)
gen subjObj_gap_w2_elections = (k_elections_w2_01 - ok_elections_w2_correct)
gen subjObj_gap_w1_elections = (k_elections_w1_01 - ok_elections_w1_correct)
gen subjObj_gap_w1_laws = (k_laws_w1_01 - ok_laws_w1_correct)
gen subjObj_gap_w2_laws = (k_laws_w2_01 - ok_laws_w2_correct)
gen subjObj_gap_w2_constitution = (k_constitution_w2_01 - ok_constitution_w2_correct)
gen subjObj_gap_w1_constitution = (k_constitution_w1_01 - ok_constitution_w1_correct)
gen subjObj_gap_w1_branches = (k_branches_w1_01 - ok_branches_w1_correct)
gen subjObj_gap_w2_branches = (k_branches_w2_01 - ok_branches_w2_correct)

*create absolute gaps by topic
gen abs_subjObj_gap_w1_laws=abs(subjObj_gap_w1_laws)
gen abs_subjObj_gap_w2_laws=abs(subjObj_gap_w2_laws)
gen abs_subjObj_gap_w1_elections=abs(subjObj_gap_w1_elections)
gen abs_subjObj_gap_w2_elections=abs(subjObj_gap_w2_elections)
gen abs_subjObj_gap_w1_constitution=abs(subjObj_gap_w1_constitution)
gen abs_subjObj_gap_w2_constitution=abs(subjObj_gap_w2_constitution)
gen abs_subjObj_gap_w1_branches=abs(subjObj_gap_w1_branches)
gen abs_subjObj_gap_w2_branches=abs(subjObj_gap_w2_branches)
gen abs_subjObj_gap_w1_parties=abs(subjObj_gap_w1_parties)
gen abs_subjObj_gap_w2_parties=abs(subjObj_gap_w2_parties)

*Table A10
*ttests for directional gap by item 
ttest subjObj_gap_w1_laws == subjObj_gap_w2_laws
ttest subjObj_gap_w1_elections == subjObj_gap_w2_elections
ttest subjObj_gap_w1_constitution == subjObj_gap_w2_constitution
ttest subjObj_gap_w1_branches == subjObj_gap_w2_branches
ttest subjObj_gap_w1_parties == subjObj_gap_w2_parties

*Table A11
*ttests for absolute gap by item
ttest abs_subjObj_gap_w1_laws == abs_subjObj_gap_w2_laws
ttest abs_subjObj_gap_w1_elections == abs_subjObj_gap_w2_elections
ttest abs_subjObj_gap_w1_constitution == abs_subjObj_gap_w2_constitution
ttest abs_subjObj_gap_w1_branches == abs_subjObj_gap_w2_branches
ttest abs_subjObj_gap_w1_parties == abs_subjObj_gap_w2_parties


**Additional Analysese in Appendix**

*For footnote 3 in Part A of Appendix re: gender coding
*re-run models with 1 = male + transgender + other; 0 = female
gen malePlus = 1 if gender_other==1
replace malePlus = 1 if gender_other==3
replace malePlus = 1 if gender_other==4
replace malePlus = 0 if gender_other==2 

* Model 1 w/ malePlus *
reg subjObj_gap_w2_5 delta_ka delta_ok ok_w1_3 overconfident courseAttendance_w2 parentEdu_w1 year_w1 gpa_combo malePlus minority i.instructor_final i.sem 
eststo m1mp

* Model 2 w/ malePlus *
reg subjObj_gap_w2_5 c.delta_ka##c.ok_w1_3 c.delta_ok##c.ok_w1_3 overconfident courseAttendance_w2 parentEdu_w1 year_w1 gpa_combo malePlus minority i.instructor_final i.sem 
eststo m2mp

* Model 3 with 3way interaction and w/ malePlus *
reg subjObj_gap_w2_5 c.delta_ka##i.overconfident##c.ok_w1_3 c.delta_ok##i.overconfident##c.ok_w1_3 courseAttendance_w2 parentEdu_w1 year_w1 gpa_combo malePlus minority i.instructor_final i.sem
eststo m3mp

esttab m1mp m2mp m3mp, b(2) se(2) ar2 bic star(+ 0.1 * 0.05)

* re-run models with 1 = female + transgender + other; 0 = male
gen femalePlus = 1 if gender_other==2
replace femalePlus = 1 if gender_other==3
replace femalePlus = 1 if gender_other==4
replace femalePlus = 0 if gender_other==1

* Model 1 w/ femalePlus *
reg subjObj_gap_w2_5 delta_ka delta_ok ok_w1_3 overconfident courseAttendance_w2 parentEdu_w1 year_w1 gpa_combo femalePlus minority i.instructor_final i.sem 
eststo m1fp

* Model 2 w/ femalePlus *
reg subjObj_gap_w2_5 c.delta_ka##c.ok_w1_3 c.delta_ok##c.ok_w1_3 overconfident courseAttendance_w2 parentEdu_w1 year_w1 gpa_combo femalePlus minority i.instructor_final i.sem 
eststo m2fp

* Model 3 with 3way interaction and w/ femalePlus *
reg subjObj_gap_w2_5 c.delta_ka##i.overconfident##c.ok_w1_3 c.delta_ok##i.overconfident##c.ok_w1_3 courseAttendance_w2 parentEdu_w1 year_w1 gpa_combo femalePlus minority i.instructor_final i.sem
eststo m3fp
esttab m1fp m2fp m3fp, b(2) se(2) ar2 bic star(+ 0.1 * 0.05)










